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ABSTRACT 

When implementing innovative teaching techniques, instructors often seek to gauge the 
success of their methods. Proposing one approach to assessing classroom innovation, this 
study examines the ability of students' ratings of engagement and instructional practices to 
predict their learning in a cooperative (team-based) framework. After identifying the factor 
structures underlying measures of student engagement and instructional practices, these 
factors were used as predictors of self-reported student learning in a general chemistry course 
delivered using a team-based learning approach. Exploratory factor analyses showed a four- 
factor structure of engagement: teamwork involvement, investment in the learning process, 
feelings about team-based learning, level of academic challenge; and a three-factor structure 
of instructional practices: instructional guidance, fostering self-directed learning skills, and 
cognitive level. Multiple linear regression revealed that feelings about team-based learning 
and perceptions of instructional guidance had significant effects on learning, beyond other 
predictors, while controlling gender, GPA, class level, number of credit hours, whether 
students began college at their current institution, expected highest level of education, racial 
or ethnic identification, and parental level of education. These results yield insight into student 
perceptions about team-based learning, and how to measure learning in a team-based 
learning framework, with implications for how to evaluate innovative instructional methods. 
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Student engagement has been regarded as a determining factor for promoting academic 
achievement (Astin, 1984; Fink, 2007; Kuh, Cruce, Shoup, Kinzie, & Gonveya, 2008; Svanum & Bigatti, 
2009). Students who engage with course content to a greater degree experience greater success (Astin, 
1984). As a result, fostering student engagement is a critical challenge for instructors regardless of their 
disciplinary focus. Cooperative learning is among the teaching methods that are claimed to promote 
student engagement (Johnson, Johnson & Smith, 2014). In the present study, a team-based learning 
approach—a form of cooperative learning (Michaelsen, Knight & Fink, 2002) —was introduced to a 
university-level general chemistry course. We examined which aspects of student engagement, and of the 
team-based instructional strategy, best predicted student perceptions of learning in the course. 
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Our goals were: 

1. to offer insight into the relationship between engagement, instructional practices, and 
perceived learning in a cooperative, team-based course environment, and 

2. to evaluate the factor structures of measures of engagement and instructional practices 
with utility for assessing innovative instructional methods. 

Given that part of the challenge for engaging students requires understanding how and why teaching 
methods designed to enhance engagement impact student learning, consideration of both the 
theoretical context of engagement and practical basis for the implementation of team-based learning is 
necessary. 

CONCEPTUALIZING STUDENT ENGAGEMENT 

In his theory of student involvement, Astin (1984) defines student involvement (i.e., 
engagement) as “the amount of physical and psychological energy that the student devotes to the 
academic experience” (p.518). The theory supports the view that learning is maximized when the 
learning environment is designed to encourage students’ active participation. Educators should, 
according to Astin, devote more time and energy to gauging and stimulating students’ motivation and 
the amount of effort they dedicate to the learning process. 

The empirical literature confirms the importance of student involvement or engagement for 
learning. For instance, Svanum et al. (2009) studied student engagement among undergraduate students 
enrolled in an upper division course in abnormal psychology. They examined whether academic course 
engagement, measured as course involvement and effort directed toward specific course components— 
such as attending lectures and completing reading assignments—predicted baccalaureate degree 
attainment and the proficiency and efficiency with which students reached this goal. They concluded 
that although—to some extent—college success relates to previously acquired skills and academic 
ability, what students do in college and the degree to which they become involved also impacts success. 
Astin’s theory (1984) of involvement along with the empirical findings of Svanum et al. (2009) suggest 
that engagement is multidimensional, containing not only skill-effort components but also an 
interpersonal component. Relatedly, other researchers argue that addressing the interpersonal 
component of engagement by implementing teaching methods that promote interactions with teachers 
and other students can stimulate and sustain engagement (Guthrie, Anderson, Alao, & Rinehart, 1999; 
Pintrich & De Groot, 1990; Skinner & Belmont, 1993). Additionally, reward, reinforcement, or 
gratification are considered crucial in the acquisition and performance of skills and knowledge (Rotter, 
1996). 

ENGAGING STUDENTS THROUGH COOPERATIVE LEARNING 

Given the stated importance of students playing an active and involved role in the learning 
process, how can student engagement be promoted in the classroom? According to a number of studies, 
active participation in learning can be accomplished through cooperative learning with group activities 
(Knabb, 2000; Rassuli & Manzer, 2005; Springer, Stanne, & Donovan, 1999). Several meta-analyses of 
the effects of cooperative learning on various outcomes (Johnson, Johnson, & Smith, 2014; Springer et 
al., 1999) reveal that cooperative learning has favorable effects on academic achievement, encourages 
positive relationships with classmates and faculty, improves psychological health and well-being, 
promotes more favorable attitudes toward learning, and increases persistence through science, 
mathematics, engineering, and technology courses and programs. According to Springer et al. (1999), 

Alvarez-Bell, R., Wirtz, D., & Bian, H. (2017). Identifying keys to success in innovative teaching: Student engagement and ^29 
instructional practices as predictors of student learning in a course using a team-based learning approach. Teaching & 
Learning inquiry, 5(2). http://dx.doi.org/! 0.20343/teachlearninqu.5.2.10 


Alvarez-Bellj Wirtz, & Bian 


there are three interrelated theoretical perspectives that describe the effects of small-group learning on 
academic achievement: motivational, affective and cognitive. 

First, from the motivational standpoint, the key to promoting a cooperative learning 
environment lies on the implementation of group goals and instructional structure that holds each group 
member accountable for learning. Group members will support each other if they think that the success 
of the group is important. Individual accountability helps to ensure that students interact and support 
one another in ways that lead to significant learning. 

Second, the affective perspective of the effects of small-group learning on academic achievement 
is grounded on the conception that group work that takes place in an encouraging atmosphere is likely to 
promote learning. When instructors facilitate supportive and open interactions among group members 
that are characterized by high-level discussions, greater conceptual understanding is achieved by the 
students (Johnson & Johnson, 1985; Smith, Johnson, & Johnson, 1981). 

Third, the cognitive perspective contends that interactions between students cause an increase 
in achievement because they provide more opportunities for information processing. While engaging in 
discussions of the course content, disagreements in reasoning will arise, multiple paths leading to the 
same solution will be uncovered, students will engage in cognitive restructuring of the material, and the 
development of higher-order thinking will result (Johnson & Johnson, 1985; Smith etal., 1981). 

TEAM-BASED LEARNING 

As one of the approaches to cooperative learning, team-based learning is an active learning 
strategy that promotes student engagement (Clark, Nguyen, Bray & Levine, 2008; Haidet, Morgan, 
O’Malley, Moran, & Richards, 2004; Prince, 2004), increases test performance (Koles, Stolfi, Borges, 
Nelson, & Parmelee, 2010), promotes positive student attitudes toward group work (Parmelee, 
DeStephen, & Borges, 2009), and develops professional competencies that are difficult to nurture or 
assess through lecture-based instruction (Beatty, Kelley, Metzger, Bellebaum, & McAuley, 2009; 
Parmelee and Michaelsen, 2010). Team-based learning is, moreover, an application-oriented 
pedagogical strategy (Thompson, Schneider, Haidet, Perkowski, & Richards, 2007) that is believed to 
promote high levels of active student participation by integrating small groups into a large class (Haidet 
& Fecile, 2006). 

Team-based learning (TBL), specifically, in contrast to other forms of cooperative, small group 
learning, makes group work the primary in-class activity (Michaelsen, Knight, & Fink, 2002) and often 
requires a change in the structure of the course to develop and take advantage of the special capabilities 
of high-performance learning teams. Fink (2002) defines team-based learning using two key ideas. One 
is that TBL involves the formation of social units that are quite different from “groups” in the sense that 
the social units are characterized by a high level of individual commitment to the welfare of the group 
and high levels of trust among the members of the group. The process by which a group of students 
becomes a “team” requires a) time for members to interact, b) scholarly resources, c) challenging tasks 
that become common goals, and d) frequent feedback on individual and group performance. A second 
key idea is that team-based learning is not simply a series of disconnected group activities, but a 
particular instructional strategy that often requires a complete course redesign. The restructuring of the 
course involves dividing the material into five to seven modules focused on the core topics of the subject. 

A three-phase sequence is incorporated within each module: preparation, application, and 
assessment, and this sequence of interdependent activities, tailored to dynamically engage students in 
the learning process, reflects the motivational, cognitive and affective perspectives of learning in a 
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cooperative context (Springer et al., 1999). Next, we describe this sequence further, conceptually and as 
implemented in a redesigned general chemistry course. 

TBL phase 1: preparation 

The goal of the preparation phase is that students familiarize themselves with the core concepts 
corresponding to a module. This was accomplished outside the classroom through completion of 
instructor-guided reading assignments or through watching instructional videos. Next, once in the 
classroom, students engaged in a readiness assurance process in which they were tested on fundamental 
concepts. Immediately after students submitted their individual answers, they repeated the same test as a 
team. Following the group quiz, teams had the opportunity to appeal wrong answers if they were due to 
ambiguity in the questions. Teams were required to provide evidence and create a convincing written 
argument as to why their answer was correct. In the final step, the instructor provided clarification of any 
misconceptions the students had. The preparation phase most closely reflects the motivational 
perspective of cooperative learning (Springer at al., 1999). 

TBL phase 2: application 

In this second phase, teams apply the concepts reviewed in the readiness assurance process to 
solve problems which require answering questions, creating explanations, or making predictions. Tearns 
submitted their answers to class problems simultaneously, and the instructor offered feedback on the 
quality of the responses. This exercise allowed students to consider different approaches to solve a 
problem, to compare their reasoning with that of classmates, and to construct new ways of thinking. The 
application phase thus most closely reflects the affective and cognitive perspectives of cooperative 
learning. 

TBL phase 3: assessment 

The sequence of interdependent activities for each module in team-based learning ends with the 
assessment phase, in which students take a comprehensive test on the module’s content. Following the 
assessment phase for the first module, teams integrated previous material with new information as they 
iteratively repeated the cycle with each subsequent module. 

OVERVIEW OF THE CURRENT STUDY 

Although studies on the use of TBL in the instruction of undergraduate communication 
(Thomas, 2012), entrepreneurship (Drummond, 2012), information technology (Jacobson, 2011), law 
(Dana, 2007), mathematics (Paterson & Sneddon, 2011), nursing (Clark, Nguyen, Bray, & Levine, 
2008), statistics (Clair & Chihara, 2012), and psychology (Grant-Vallone, 2011; Thomas & 

McPherson, 2011) courses have been reported, the use of TBL in the instruction of an undergraduate 
general chemistry course has not been highlighted. In this study, following a full course redesign to 
integrate TBL, we surveyed general chemistry students’ perceptions of engagement, instructional 
practices, and learning. Students’ responses were analyzed and, after evaluating their factor structure and 
reliability, the predictive value of these student engagement constructs and instructional practices on 
self-reported learning was examined. 

METHODOLOGY 

An online survey was used to assess students’ perceptions of engagement, instructional practices, 
and learning outcomes. Exploratory factor analyses using principal axis factoring with Direct Oblimin 
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rotation were conducted on the engagement and instructional practices items to identify their factor 
structures. The relationship between engagement and instructional practices on student learning was 
examined by employing multiple linear regression. 

Participants 

All undergraduate students (N = 111) enrolled in a section of a 14-week General Chemistry II 
course at a 4-year degree public university in eastern North Carolina were invited to participate in this 
study. Response rates to the online survey questions ranged from 108/111 (teamwork and feelings 
items) to 102/111 for the class status item (Table l). The demographics of our non-randomized sample 
are representative of the students in the course under study. Nearly 57% of the participants were 
women; 71% of participants classified themselves as White. The majority of the participants were full¬ 
time students (96%) and most of them had a self-reported CPA 3.0 or above (86%). 

Students were invited to participate by email and provided electronic consent by answering the 
first question in the survey (‘... Do you consent to participate in this study?’). Additionally, the 
instructor read an invitation to participate aloud during class. Participation was voluntary and no 
monetary compensation was provided, but extra credit was granted. The research study was approved by 
the lead author’s Institutional Review Board. 


Table 1. Descriptive statistics 


Variable 

n 

% 

Mean* 

S.D. 

Gender 

Male 

46 

43.4 

- 

- 

Female 

60 

56.6 

- 

- 

Ethnicity 

Asian 

9 

8.6 

- 

- 

Black or African American 

10 

9.5 

- 

- 

Hispanic or Latino 

9 

8.6 

- 

- 

White 

75 

71.4 

- 

- 

Other 

2 

1.9 

- 

- 

Class 

Freshman 

45 

44.1 

- 

- 

Sophomore 

34 

33.3 



Junior 

17 

16.7 

- 

- 

Senior 

6 

5.9 

- 

- 

Engagement 

T eamwork 

108 


4.15 

.67 

Reaching out 

106 


2.20 

1.08 

Feelings 

108 


3.71 

.76 

Personal investment 

106 


3.95 

.67 

Instructional practices 

Cognitive 

107 


3.80 

.76 

Opportunities 

105 


4.50 

.77 

Guidance 

106 


4.31 

.84 

Self-reported student learning 

106 


4.04 

.85 
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*Scale of 1 to 5 with 5 indicating greater levels of the characteristics being measured. Totals that are less than the 
total number of participants (N = 108) reflect non-responses. 

Materials 

The survey consisted of 81 items, including four sections: demographic information (12 items), 
student engagement (43 items), instructional practices (20 items), and self-reported student learning (6 
items). 


Demographics 

Participants were asked to report their gender, racial/ethnic identification, GPA, class level, 
whether they were full-time students, number of credit hours for which they were enrolled, at what 
institution they began college, highest level of education they expected to complete, highest level of 
education completed by either parent, whether they were members of a sorority or fraternity, whether 
they had been diagnosed with any disability or impairment, and, if they had been diagnosed with a 
disability or impairment, the type of disability or impairment. 

Studen t engagemen t 

The student engagement questions were created by adapting selected items from the National 
Survey of Student Engagement (NSSE) instrument (2012). The items chosen were intended to 
measure primarily the behavioral dimension of engagement and to identify effective educational 
practices that support such behaviors. Since the NSSE is designed to reflect general engagement in 
school, most of the items were reworded in order to make them specific to the General Chemistry II 
course and to the TBL teaching strategy. In addition, items reflecting psychological components of 
engagement were also incorporated in the survey in order to gain better understanding of all aspects of 
engagement that may promote student learning. These items were adapted from Louis and Schreiner’s 
Engaged Learning Index (Schreiner & Louis, 2011). 

The resulting student engagement scale included 43 items reflecting how often students 
engaged in collaborative learning (e.g., “Asked questions during group discussions”; 1 [never] to 5 [ all of 
the time]), students’ feelings about the particular course (e.g., “How often you felt that the interaction 
with your teammates helps you increase understanding of the course material”; 1 [never] to 5 [all of the 
time]), students’ self-regulation (e.g., “Identified key information from reading assignments, videos, or 
PowerPoint lecture slides”; 1 [very much] to 5 [not at all]), and the number ofhours students allocated 
to academic, personal, and co-curricular activities in a typical 7-day week (0hours, 1-5hours, 6-10 
hours, 11-15hours, 16-20hours, 21-25hours, 26-30hours). These items are indirectly related to the 
motivational, affective, and cognitive perspectives of the effect of small-group learning on academic 
outcomes as they reflect students’ behaviors that may influence their learning outcomes. 

Instructional practices 

Next, participants responded to 20 items involving students’ perceptions of the guidance and 
feedback provided by the instructor (e.g., “Clearly explained course goals and requirements”; 1 {[very 
much] to 5 [not at all] ), the extent to which the instructor provided opportunities for the development 
of self-regulated learning skills (e.g., “Providing opportunities for students to take initiative and 
responsibility for their own learning”; 1 [verymuch] to 5 [not at all]), and how much different levels of 
learning were emphasized in the particular course (e.g, “Analyzing the basic elements of an idea, 
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experience, or theory such as examining a specific issue/problem in depth and considering its 
components”; 1 [very much] to 5 [not at all]). 

Perceived learning 

To assess students’ perceptions of learning outcomes, six items selected from the NSSE survey 
instrument (2012) were used. The items gathered students’ perceptions of their gains in higher-order 
thinking skills (“Thinking critically and analytically”, “Analyzing numerical information”), work-related 
competencies (“Expressing ideas clearly and effectively”), group skills (“Working effectively with 
others”), and self-directed learning skills (“Learning effectively on your own”). All items used a five- 
point scale: 1 (very much) to 5 (not at all). Internal consistency of the self-reported student learning 
items was analyzed using Cronbach’s reliability test (alpha = .947). 

Data Analysis 

Data analyses was performed using the Statistical Package for the Social Sciences (SPSS) version 
22.0. All statements in the survey instrument utilizing 1 [very much] to 5 [not at all] response scales 
were reverse-scored prior to analysis in a manner consistent with higher numbers equaling more of the 
attribute (e.g. higher engagement, higher effectiveness of instructional strategy and greater learning 
outcomes). Exploratory factor analysis (EFA) was conducted on the engagement and instructional 
practices items to collapse the large number of variables contained in each set into a few interpretable 
underlying factors. A principal axis factoring analysis with Direct Oblimin rotation was undertaken to 
identify the factor structure for each scale. Parallel analysis (O’Connor, 2000) was used to determine the 
number of factors for each scale. Total scores for each factor obtained from EFA were computed. As for 
item performance, reliability analysis was conducted for each scale. Bivariate correlation analyses were 
performed to assess collinearity among potential predictors of self-reported student learning. In order to 
determine which components of engagement and instructional practices had impact on self-reported 
student learning, while controlling gender, GPA, class level, number of credit hours, whether students 
began college at the current institution, expected highest level of education, racial or ethnic 
identification, and parental highest level of education, multiple linear regression analysis was performed. 

RESULTS 

Exploratory factor analysis: student engagement 

Principal axis factoring analysis with Direct Oblimin rotation was performed to identify the 
underlying factors of the engagement scale. The Kayser-Meyer-Olkin measure of sample adequacy was 
0.78, suggesting that the data were sufficient relative to the number of items in the scale (Hutcheson & 
Sofroniou, 1999). Barlett’s test of sphericity, % 2 (378) = 2093.12, p < .001, showed that the correlations 
between the items were appropriate for factor analysis (Field, 2009). A scree plot from parallel analysis 
indicated a four-factor structure of student engagement. The cut-off criterion for item loadings was set to 
.40 (Stevens, 1992). Items with factor loadings smaller than .40 were removed. After examining item 
performance using item-total correlations and Cronbach’s alpha, five items showing poor performance 
were removed (‘Worked with other students outside of class time to complete class assignments or study 
for an exam’, ‘Included diverse perspectives in course discussions or assignments’, ‘Felt overwhelmed by 
the amount and intensity of the course work’, ‘Felt that you are not good at chemistry and that there is 
nothing you can do to become good at it’, and ‘Completed homework by the due date’) and principal 
axis factoring was re-run. 
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All four factors had moderately high factor loadings (Table 2; Osborne & Costello, 2009). The 
first factor consisted of six items and was labeled ‘feelings about the course.’ These items reflect students’ 
positive attitudes about the team-based learning pedagogical strategy and their interactions with 
classmates. The second factor, which consisted of seven items, was labeled ‘reaching out for academic 
assistance’ because these items reflected seeking additional help from or further discussion with 
instructors, teaching assistants, and tutors. The name given to the third factor, which consisted of eight 
items, was ‘personal investment in the learning process’ because it includes actions taken by students 
that help them achieve their educational goals as well as reflections on what they learned. The fourth 
factor, ‘teamwork,’ consisted of six items indicating students’ active participation in collaborative 
learning. 


Table 2. Four factor solution of engagement items 



Factor 1 

Factor 2 

Factor 3 

Factor 4 

Items 

(Feelings) 

(Reaching out) 

(Personal 

Investment) 

(Teamwork) 

Felt excited about the team-based learning 
classroom 

.80 




Felt fascinated about the course content 

.72 




Felt that chemistry is challenging for you but 
that with hard work and a realistic study 
strategy, you can excel 

.42 




Felt that you are a valuable member of your 
learning team 

.42 




Felt that the interaction with your teammates 
helps you develop your communication skills 

.78 




Felt that the interaction with your teammates 
helps you increase understanding of the 
course material 

.75 




Attended weekly review sessions 


.57 



Attended Tutoring Center tutorial sessions 


.51 



Attended instructor’s office hours 


.82 



Made an appointment with your instructor, a 
tutor, or teaching assistant to get assistance 


.82 



Talked about career plans with your 
instructor* 


.77 



Discussed course topics, ideas, or concepts 
with your instructor outside of class* 


.87 



Discussed your academic performance with 
your instructor* 


.91 



Fully completed the pre-class reading 
assignments 



.82 


Reviewed the completed pre-class reading 
assignments before attending the 
corresponding class meeting 



.82 


Watched videos suggested by your instructor 
as a way to prepare for class meetings 



.60 
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Identified key information from reading 
assignments, videos, or PowerPoint lecture 
slides* 

.72 


Examined the strengths and weaknesses of 
your own views on a topic or issue* 

.51 


Realized that you have learned something 
that changed the way you understand an 
issue or concept* 

.52 


Taken advantage of available learning 

resources 

.53 


Summarize what you learned in class or from 
course material provided* 

.48 


Asked questions during group discussions* 


.83 

Contributed to group discussions 


.85 

Worked with other students during class time 


.63 

Explained course material to one or more 
classmates in the classroom* 


.69 

Had a classmate explain course material to 
you in the classroom 


.78 

Reflected on feedback provided by your 
teammates regarding your contribution to 
team work 


.56 


*ltems used with permission from The College Student Report, National Survey of Student Engagement, 
Copyright 2001 -17 The Trustees of Indiana University. 

To examine the reliability of the identified factors, internal consistency was assessed using 
Cronbach’s alpha. Feelings about the course (a = .88), reaching out for academic assistance (a = .91), 
personal investment in the learning process (a = .88), and teamwork (a = .90) had strong internal 
consistency. 

Exploratory factor analysis: instructional practices 

For items grouped in the instructional practices set, the Kaiser-Meyer-Olkin measure of sample 
adequacy was .86, indicating that the data were sufficient relative to the number of items in the scale 
(Hutcheson et ah, 1999). Barlett’s test of sphericity, % 2 (l90) = 2293.545, p < .001, showed that the 
correlations between the items were appropriate for factor analysis (Field, 2009). A scree plot from 
parallel analysis was also used to determine the number of factors. Only factors for which the eigenvalues 
were greater than 1.00 (Pett, Lackey, & Sullivan, 2003) were retained. The eigenvalues for the first three 
factors were 9.49, 3.25, and 2.25; however, the fourth eigenvalue (0.584) was lower than 1.00, which 
indicated that only three factors should be retained for analysis. The three-factor structure that was 
identified is presented in Table 3. 

The factor loadings for the instructional practices set were robust. The factors were named 
instructional guidance (Factor l), cognitive level (Factor 2), and fostering self-regulation (Factor 3). 
The instructional guidance factor contained nine items reflecting the instructor’s guidance and feedback 
as well as the clarity and organization of learning activities. The six items comprising the cognitive level 
factor address the degree to which students perceived different levels of knowledge and cognitive 
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processes as being nurtured by the instructional practices used. The third factor, fostering self¬ 
regulation, consists of six items that reflect opportunities provided for developing self-regulated learning 


skills. 

Table 3. Three factor solution for instructional practices items 


Items 

Factor 1 

Factor 2 

Factor 3 


(Instructional 

(Cognitive level) 

(Fostering self¬ 


Guidance) 


regulation) 

Clearly explained course goals and requirements* 

.92 



Clearly described what students are expected to 
do to prepare for each class meeting 

.90 



Clearly described the activities that will take place 
during each class session and the order in which 
they will occur 

.86 



Used examples or illustrations to explain difficult 
points* 

.89 



Incorporated class activities that promote self¬ 
reflection 

.75 



Provided guidance and feedback during the 
active-learning class activities 

.93 



Provided prompt feedback on tests or completed 
assignments* 

.81 



Encouraged collaborative work 

.72 



Provided advice on how to prepare for an exam 

.86 



Memorizing facts, ideas, or methods from your 
course and readings so you can repeat them in 
almost the same form* 


.55 


Analyzing the basic elements of an idea, 
experience, or theory such as examining a specific 
issue/problem in depth and considering its 
components* 


.84 


Reaching conclusions based on your own analysis 
of numerical information (numbers, graphs, 
statistics, etc.)* 


.86 


Evaluating a point of view, decision, or 
information source (i.e., examining how others 
gathered and interpreted data and assessing the 
accuracy of their conclusions)* 


.64 


Applying theories or methods to practical 
problems or new situations* 


.83 


Synthesizing and organizing ideas, information, 
or experiences into new, more complicated 
interpretations and relationships (i.e., forming a 
new idea or understanding from various pieces of 
information or connecting ideas from your course 
to your prior experiences and knowledge* 


.87 
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Providing opportunities for students to take 
initiative and responsibility for their own learning 

.93 

Providing support to help students succeed 
academically 

.92 

Encouraging students to identify and use learning 

.92 

support services 

Providing opportunities for students to build 
collaboration and communication skills 

.97 

Providing opportunities for students to identify 
and address gaps in their learning 

.85 


*ltems used with permission from The College Student Report, National Survey of Student Engagement, 
Copyright 2001 -17 The Trustees of Indiana University. 

Cronbach’s alphas for all three factors—instructional guidance (a = .96), cognitive level (a = .90), and 
fostering self-regulation (a = .97)—confirmed strong internal consistency. 

Multiple linear regression 

Each of the student engagement factors (feelings toward the course, reaching out for academic 
assistance, personal investment in the learning process, and level of participation in teamwork) and each 
of the instructional practices factors (instructional guidance, cognitive level, and opportunities for self¬ 
regulation) were used in a standard regression analysis to predict student self-reported learning. The 
factor scores were computed by averaging item total scores (see Table l). Bivariate correlation analyses 
indicated that correlation coefficients between independent variables were below the threshold | r | > .7 
(Table 4) that is considered an appropriate indicator for when collinearity begins to severely distort 
model estimation and subsequent prediction (Dormann et al., 2013; Meyers, Gamst, & Guarino, 2006). 


Table 4. Correlations for engagement, instructional strategy ( n = 105) 


Variables 

1 

2 

3 

4 

5 

6 

7 

1. Feelings 

- 

.20 

.45 

.61 

.25 

.46 

.57 

2. Reaching out 


- 

.30 

-.04 

.26 

.08 

-.16 

3. Personal investment 



- 

.46 

.53 

.51 

.36 

4. T eamwork 




- 

.27 

.27 

.59 

5. Cognitive Level 





- 

.33 

.21 

6. Guidance 






- 

.56 

7. Fostering self-regulation 







- 


Demographic characteristics such as students’ gender, race, intellectual ability (as measured by 
GPA), their educational aspirations, and the educational level attained by their parents are among the 
factors presumed to affect students’ learning (Artino, 2007; Astin, 1993; Friday, et ah, 2006; Oakes, 

1990; Pascarella & Terenzini, 2005). As this study is centered on investigating the relationship between 
learning and a particular set of independent variables (teamwork, reaching out, feelings about the course, 
personal investment, levels of learning, opportunities for developing self-regulated skills, instructor’s 
guidance), to adjust for any possible influence by the demographic characteristics included on our 
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survey, we used them as control variables for the multiple linear regression analysis. Male served as the 
reference group for gender. GPA was coded as three dummy variables, with GPA =3.0 serving as the 
reference group (see Table 5). Class level was coded as three dummy variables with freshmen serving as 
the reference group. 

The full multiple linear regression model with independent variables and control variables was 
significant, F( 18, 82) = 9.85, p < .001 and produced an adjusted R 2 = .68, which indicates that the 
predictors in this model explained 68% of the variance of student self-reported learning. Only two 
predictors, feelings about the course ( B= .63, p= .00) and instructional guidance ( B= .24 ,p = .01), 
emerged as statistically significant (see Table 5). Using the standardized coefficients to compare the two 
significant predictors, the following order of importance is noted: feelings (.56), and guidance (.24). 
Higher values for each of these predictors are associated with greater student learning while controlling 
other predictors in the model. 


Table 5. Linear regression results (unstandardized slope, standard error, standardized betas and probability values) 


Predictors 

B 

SE 


P 

Feelings 

.63 

.11 

.56 

.00* 

Reaching out 

-.01 

.07 

-.01 

.94 

Personal investment 

.26 

.13 

.20 

.054 

Teamwork 

-.18 

.13 

-.14 

.16 

Cognitive Level 

.12 

.09 

.10 

.19 

Guidance 

.24 

.09 

.24 

.01* 

Fostering self-regulation 

-.04 

.11 

-.03 

.75 

Gender: Female vs. Male 

.20 

.12 

.11 

.11 

GPA 

3.7-4.0 

.28 

.22 

.14 

.20 

33-3.6 

.10 

.19 

.05 

.61 

3.0-3.2 

.25 

.19 

.13 

.20 

Class Level 

Sophomore 

.29 

.14 

.16 

.045* 

Junior and Senior 

-.01 

.23 

-.01 

.97 

Credit hours 

-.09 

.04 

-.17 

.02* 

Begin college here 

-.03 

.22 

-.01 

.88 

Expected highest level of education: Master or higher 

-.08 

.14 

-.04 

.55 

Race: White vs. others 

.01 

.13 

.01 

.92 

Highest level of education completed by parents 

-.01 

.04 

-.02 

.82 


The reference group for gender is male. 

The reference group for GPA is 3.0. 

The reference group for class level is freshmen. 

The reference group for expected highest level of education is bachelor or lower. 
The reference group for begin college here is began college elsewhere. 

*p< .05. 
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DISCUSSION 

What features of student engagement and of the instructional practices used in this study best 
predict student learning? Our data suggest the answer is twofold: students’ feelings about the course (P 
= .56) and students’ perceptions of the extent to which instructional guidance was provided (P = .24) 
both significantly predicted self-reported learning. Closely following these two factors was a third— 
personal investment in the learning process (P = .20, p = .054)—which we also consider in the following 
discussion. 

Feelings about the course 

Our findings suggest that the more students felt excitement toward team-based learning, the 
greater their perceptions that team-based interactions were helpful, and the more fascinated they 
reported being by the course content, the more they judged themselves as having gained higher order 
thinking skills, work-related competencies, group skills and self-directed learning skills. There are at least 
two interpretations of the causality of this relationship. The first is that student responsiveness to the 
course structure—characterized by positive feelings about a team-based approach to chemistry—leads 
to the perception of superior learning. The second is that students who felt they learned more came to 
feel positively toward TBL and the course more generally (i.e., the reverse causal interpretation). 

Though the present study is cross-sectional and lacks a non-TBL comparison group, and thus cannot 
definitively address causality or the unique effect of course design on our outcomes, we view it as entirely 
possible that the inherent features of team-based learning served as a vehicle for improving students’ 
feelings toward the course, and—as our analysis shows—more positive feelings toward a general 
chemistry course lead to greater learning. To the extent that students experienced substantial cognitive 
engagement, they consequently developed more positive attitudes toward learning chemistry. TBL is 
designed to promote high levels of student interactivity, in part, by providing an atmosphere in which 
learning is fueled not only by peer support but also by peer challenge. Students also build relationships 
with their teammates that go beyond mere academic work. Within each team, acquaintances may evolve 
into friendships. The positive interpersonal relationships fostered by TBL can serve as a mechanism by 
which TBL increases students’ desire and ability to learn. As evidenced by the statement below, made by 
a student, the ongoing social interactions that are possible by being with the same group of peers for the 
whole academic term gives rise to a supportive network that stimulates learning. 

It (TBL) helps build up my confidence and it helps me see how the other team members solve 

this problem. By socializing together, we learn more. 

Our interpretation is consistent with Springer, Stanne and Donovan’s (1999) affective 
perspective on the effects of small-group learning on academic achievement. According to the affective 
viewpoint, group work that takes place in an encouraging atmosphere is likely to promote learning. In 
the TBL framework used in this redesigned course, students had the opportunity to feel that they were 
valuable members of a group and that the interaction with their peers, teaching assistants and instructor 
helped them enhance their feelings of self-efficacy in chemistry. This outcome is consistent with 
students having more positive attitudes toward learning chemistry and, consequently, experiencing 
greater cognitive engagement in a TBL context. These findings are also consistent with the control-value 
theory of achievement emotions (Pekrun & Perry, 2014). According to this theory, emotions influence 
students’ academic performance and are linked to motivation, use of learning strategies, and self- 
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regulation. Providing students with learning environments that stimulate autonomy and cooperation are 
presumed to foster students’ sense of competence, which positively impacts their emotions (Pekrun, 
2006). 


Instructional guidance 

Second only to their feelings about the course, the perceived degree of instructional guidance 
was significantly and positively related to students’ gains in higher-order thinking skills, group skills and 
self-regulatory learning skills. These results suggest that students’ learning can be positively impacted by 
clearly explaining course goals and expectations, using illustrations to explain difficult points, providing 
immediate and frequent feedback on course assignments and assessments, incorporating activities that 
promote self-reflection, and encouraging collaborative work. While these instructional practices are not 
unique to team-based learning, the implementation of practices such as providing immediate and 
frequent feedback, incorporating activities that promote self-reflection, and encouraging collaborative 
work is facilitated in the team-based learning framework. As proposed by the cognitive perspective of the 
effects small-group learning has on academic achievement (Springer et ah, 1999), interactions between 
students increase achievement because they provide more opportunities for information processing. 

The discussions that take place within each team as well as among different teams expose students to 
ideas that challenge their assumptions and ways of thinking. When students are confronted with the 
alternative viewpoints of team members or other teams, they get immediate feedback and are placed in a 
position in which they have to think critically and elaborate on their views to defend them. By going 
through this process, students gain a deeper conceptual understanding. 

The finding that instructional guidance is a significant predictor of students’ learning can be 
further understood by taking into consideration that, in addition to containing subject-specific 
terminology, textbooks—particularly in chemistry—often require that readers make inferences to fill 
conceptual gaps, making comprehension of the course material challenging, even for skilled readers 
(Zwaan & Singer, 2003). Researchers have suggested that to overcome comprehension difficulties, 
instructors should help students bridge the gaps between the content presented and text readings (Best, 
Rowe, Ozuru, & McNamara, 2005). Our findings indicate that instructor interaction and guidance 
inside and outside of the classroom have an important impact on students’ learning outcomes (cf. 
Cabrera, Colbeck, & Terenzini, 2001). 

Additional predictors 

The personal investment factor, which reflects whether students engaged in a range of 

preparatory activities and interacted in meaningful ways with course material, each reflecting a 

commitment to the learning process, had an effect size on student learning that was only slightly behind 

instructional guidance. Several researchers have suggested that collaborative learning helps students 

develop and use the self-regulatory skills that might lead to these behaviors (Hadwin, Jarvela, and Miller, 

2011; Kitsantas, Zimmerman and Cleary, 2000). That is, team-based learning is a form of collaborative 

learning that incorporates many of the processes that help students acquire skills described in 

Zimmerman’s (2002) self-regulated learning model. Team problem-solving activities provide 

opportunities for students to observe, evaluate and monitor the work of peers as well as their own work, 

allowing them to acquire and emulate those skills and behaviors that are associated with greater learning 

(Kitsantas, Zimmerman and Cleary, 2000). Planning and allocating resources to meet task demands 

(Pintrich, 1990) is required of students in team-based learning, and includes those engagement activities 

comprising the personal investment factor: e.g., fully completing pre-class reading assignments, 
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reviewing readings before attending class meetings, watching instructional videos suggested by the 
instructor as a way to prepare for class sessions, identifying key information, evaluating one’s own views 
of a topic, summarizing what is learned during class activities or from course material provided, and 
taking advantage of available learning resources. 

All other engagement factors (reaching out, teamwork) and instructional practices (cognitive 
level, fostering self-regulation) were non-significant predictors of self-reported learning. In some cases— 
such as for the teamwork factor—these findings are surprising and contrary to much of the research on 
team-based learning, which has generally reported positive associations between teamwork and learning 
(Beatty, Kelley, Metzger, Bellebaum, & McAuley, 2009; Koles, Stolfi, Borges, Nelson, & Parmelee, 

2010; Parmelee & Michaelsen, 2010). The present study found that feelings about team-based learning 
are a greater predictor of learning than the behaviors comprising our teamwork factor (e.g., asking 
questions during and participating in group discussions). 

Though not the primary focus of the present study, examining our control variables revealed a 
pair of significant associations. First, sophomore (second-year) students reported greater learning 
outcomes than freshmen, but learning outcomes for juniors and seniors were not statistically different 
from those perceived by freshmen. Based on our data, we do not have a plausible explanation for these 
findings. More research is needed to examine how class level is associated with learning in a team-based 
learning environment. 

Second, the negative standardized coefficient value for the number of credit hours for which 
students were enrolled (P = -. 17, p = .02) implies that self-reported learning decreases with increasing 
number of credit hours. This finding contradicts previous research indicating that students who take 
heavier academic loads achieve greater academic success (Abrams & Jernigan, 1984). These 
incongruent findings suggest that the relationship between academic load and learning outcomes is 
complex and that further research investigating this association using a direct measure of learning, such 
as GPA, is deemed necessary. 

Limitations 

Considering that we did not seek to gather information for academic program assessment or 
accountability purposes, the study relied on self-report measures of gains in learning outcomes. 
According to Pike (2011), self-report data is appropriate for scholarly research in which the goal is to 
identify relationships among measures that are grounded in a theoretical framework—such as the 
relationships between those learning outcomes that are believed to be promoted through collaborative 
learning, selected student engagement constructs, and instructional practices. Further support for the 
validity of self-reports of educational outcomes as indicators oflearning is provided by Anaya (1999). 

She conducted a direct comparison of college grade-point average (GPA), standardized test scores 
(GRE), and self-reports oflearning gains, using a nationally representative sample of 2,289 students. Her 
results showed that self-reports of educational outcomes exhibited validity and produced comparable 
conclusions, regarding the impact of college on student learning, to those based on standardized tests 
and grade-point average measures. Thus, while objective learning outcomes are desirable when available, 
self-reports oflearning gains are reasonable and efficient proxies. 

Another limitation of this study is that the study was conducted in one institution using a small, 
non-randomized sample of students enrolled in a general chemistry course. Though our findings are 
consistent with past research (Fassinger, 1995; Nunn, 1996), the study needs to be replicated to 
determine whether the relationships found will hold true for other freshmen-level science courses that 
are delivered using similar collaborative, learner-centered teaching methods. 
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CONCLUSIONS 

In this study, we set out to ascertain the predictive value of selected student engagement 
constructs and instructional practices on self-reported outcomes in a general chemistry course delivered 
using a team-based learning approach. Our study supports the notion that students’ learning is primarily 
influenced by their feelings about their learning environment. The team-based learning approach used in 
this study provides an environment that encourages collaboration within each learning team and 
between different teams. Participating in team discussions during the readiness assurance process (i.e., 
team quizzes) and during problem-solving activities helps students realize that they can achieve higher 
levels of understanding of the course material through cooperative learning. The more students 
indicated feeling that the TBL approach was beneficial, the more they likely perceived this pedagogical 
method as beneficial to their learning needs, thus developing a positive view of the learning 
environment. Our findings corroborate past research; for example, Fassinger’s (1995) and Nunn’s 
(1996) studies reported that among the strongest predictors of classroom participation were students’ 
perceptions that the classroom environment is supportive. 

The second most important factor predicting students’ perceived learning is their view of the 
instructor’s interest and investment in their learning. An instructor who clearly explains course goals and 
expectations, uses illustrations to explain difficult points, provides immediate and frequent feedback on 
course assignments and assessments, incorporates activities that promote self-reflection, and encourages 
collaboration among students, positively relates to students’ perceived learning outcomes. These 
findings are in line with evidence from other research showing that instructor-student interaction, 
guiding learning rather than lecturing, and giving specific and timely feedback and encouragement 
positively influence student development (Cabrera et al., 2001). Likewise, Chickering and Gamson 
(1987) advocated that instructors who encourage contact between students and faculty, develop 
reciprocity and cooperation among students, encourage active learning techniques, integrate prompt 
feedback, emphasize time on task, hold high expectations, and promote respect for diverse talents and 
ways of learning promote high levels of student engagement and satisfaction, learning, development on a 
variety of dimensions, and persistence. 

The findings in this study corroborate the theoretical-affective perspective proposed by Springer 
(1999) to describe the effects of small-group learning on academic achievement. Positive learning 
outcomes can be enhanced when instructors focus on being facilitators for learning rather than on 
serving as indubitable experts on the subject matter, concentrate time and energy on providing a 
learning environment and strategies that allow students to take active roles in the learning process and to 
be accountable for their own and their peers’ learning. Since—according to our findings—the constructs 
that best predict students’ learning outcomes are their feelings about the learning environment and 
extent of perceived instructional guidance, we believe that instructors should give careful consideration 
to these factors when developing, implementing, and refining team-based learning approaches. 
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